Image processing apparatus, image processing method and recording medium

ABSTRACT

An image processing apparatus that selects an image output mode from one of a plurality of output modes, the image processing apparatus including: an output mode selection unit that selects the image output mode by sequentially determining whether or not to select an image output mode in accordance with an output mode order allotted to the plurality of output modes, based on a ratio of a classified number of blocks to a reference number of blocks, wherein in a first determination in the output mode order, the reference number of blocks is a total number of blocks of the image, and in a second or subsequent determination in the output mode order, the reference number of blocks is a value that is obtained by subtracting the classified number of blocks in a preceding determination in the output mode order from the reference number of blocks in the preceding determination.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from Japanese Patent Application No.2010-293577 filed on Dec. 28, 2010, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

Aspects of the invention relate to a technology of classifying an imagewith respect to color characteristics and selecting an image outputmode, based on image data indicating the image, and particularly, to atechnology of preventing a misclassification of an image.

BACKGROUND

In fields of printers and displays, an image, i.e., text, photograph,graphic and the like, is output by printing the image on a printingmedium or by displaying the image on a display. An image that is anoutput target, hereinafter, referred to as “target image”, has variouscolor characteristics. The target image may be a monochrome image or acolor image.

Adding a remark to the definition of the term “image” throughout thespecification, when a document of one page is divided into an objectpart, i.e., text, photograph, graphic and the like, and a backgroundpart, i.e., background color part, the term “image” should not beinterpreted to mean a term indicating only the object part and should beinterpreted to mean a term indicating the whole document including thebackground part. According to this definition, the term “image of onepage” means “an image that is displayed on the whole document of onepage.”

In the fields of printers and displays, when a target image i.e.,document, is actually a monochrome image, it is ideal to output thetarget image in a monochrome output mode after distinguishing the targetimage as a monochrome image. Likewise, when a target image is actually acolor image, it is ideal to output the target image in a color outputmode after distinguishing the target image as a color image.

Accordingly, in the field of image outputting, based on image dataindicating an image of one page, an image output mode for outputting theimage has been selected page by page as one of a plurality of outputmodes including a monochrome output mode and a color output mode.

According to a technology disclosed in related-art, based on image dataindicating a target image, color pixels are extracted from a pluralityof pixels configuring the target image and are counted. When a number ofcounts exceed a threshold value, the target image is distinguished as acolor image. Also, the target image of one page is divided into aplurality of blocks, blocks including color pixels are extracted fromthe plurality of blocks and a number of the color pixels in theextracted blocks are counted as the number of color pixels.

According to a technology disclosed in another related-art, the imagedata, which is obtained by reading a document, is divided into aplurality of blocks and it is determined whether each block is a colorblock or a monochrome block. The number of the color blocks is counted.When the number of counts exceeds a threshold value, it is determinedthat the image of the document is a color image. On the other hand, whenthe number of counts is the threshold value or smaller, it is determinedthat the image of the document is a monochrome image.

The inventors of the invention performed various researches regardingthe image processing method of selecting, based on image data indicatingan image, an image output mode for outputting the image from one of aplurality of output modes including a monochrome output mode and a coloroutput mode. As a result, the inventors found the following findings,which will be specifically described later.

When a color part exists in a document to be read, it is natural that areading result of the document includes the color pixels. However, eventhough a document to be read does not include a color part, the readingresult of the document may include the color pixels due to a colorshift, which will be specifically described later, at the time ofreading an image.

When the color shift occurs, the document may be misclassified into acolor image and thus a color output mode may be erroneously selected,even though the document should be normally classified into a monochromeimage and a monochrome output mode should be selected. When performingthe image processing, it is important to prevent the misclassificationof the target image and the erroneous selection of the image output modedue to the misclassification.

Hereinafter, a situation where a document is classified into a colorimage when a ratio of the number of color pixels of a plurality ofpixels configuring a target image, i.e., a document of one page, to anumber of total pixels configuring the document is larger than athreshold value is considered.

In this case, the classification result of the document depends on aratio of the number of color pixels, which is a ratio of the number ofcolor pixels to the total number of pixels. In the meantime, regarding acase where a color object, i.e., text, photograph, graphic and the like,exists on a document, when a first color object is displayed on a firstdocument in a large size, the ratio of the number of color pixels islarge. However, regarding a case where a second color object isdisplayed on a second document in a small size, for example, where thesecond color object, which is the first color object being reduced, isdisplayed on the second document having the same size as the firstdocument, the ratio of the number of color pixels is small.

Since the two documents both have the color objects, the two documentsshould be classified into a color image. However, in some cases, whenthe color object is displayed in a large size on the document, thedocument is classified into a color image, but when the color object isdisplayed in a small size on the document, the document may beerroneously classified into an image other than a color image, forexample a monochrome image, due to a reduction in the ratio of thenumber of color pixels. Accordingly, an output mode other than the coloroutput mode may be erroneously selected. Therefore, when performing theimage processing, it is also important to prevent the misclassificationof the target image and the erroneous selection of the image output modedue to the misclassification.

Based on the above-described findings, aspects of the invention providea technology of classifying an image with respect to colorcharacteristics thereof, based on image data indicating the image, andselecting an image output mode, and an object of the invention is toeasily prevent a misclassification of color characteristics of an image.

SUMMARY

The invention provides following aspects, respectively. Each aspect isdistinguished by a paragraph and each paragraph is accompanied by anumber. Each paragraph is described in a format of depending on thenumber of the other paragraph, as required. However, by describing eachparagraph in a format of depending on the number of the other paragraph,it is possible to enable the technical features described in therespective paragraphs to be appropriately independent each other.

According to an aspect of the invention, there is provided an imageprocessing apparatus that selects, based on data of an image that isacquired, an image output mode for outputting the image from one of aplurality of output modes. The image processing apparatus includes apixel classification unit, an image division unit, a blockclassification unit and an output mode selection unit. The pixelclassification unit classifies each pixel of the image into one of aplurality of pixel categories based on the image data of each pixel. Theimage division unit divides the image into a plurality of blocks, theplurality of blocks each including a plurality of pixels adjacent toeach other. The block classification unit classifies each of theplurality of blocks into one of a plurality of block categories based onthe pixel category of each of the plurality of pixels belonging to eachof the plurality of blocks. The output mode selection unit selects theimage output mode by sequentially determining whether or not to selectan image output mode as one of the plurality of output modes inaccordance with an output mode order allotted to the plurality of outputmodes, based on a ratio of a classified number of blocks, which is anumber of blocks classified into each block category, to a referencenumber of blocks. In a first determination in the output mode order bythe output mode selection unit, the reference number of blocks is atotal number of the plurality of blocks of the image, and in a second orsubsequent determination in the output mode order by the output modeselection unit, the reference number of blocks is a value that isobtained by subtracting the classified number of blocks in a precedingdetermination in the output mode order by the output mode selection unitfrom the reference number of blocks in the preceding determination inthe output mode order by the output mode selection unit.

According to another aspect of the invention, there is provided an imageprocessing method for selecting, based on data of an image that isacquired, an image output mode for outputting the image from one of aplurality of output modes, the image processing method comprising:classifying each pixel of the image into one of a plurality of pixelcategories based on the image data of each pixel; dividing the imageinto a plurality of blocks, the plurality of blocks each including aplurality of pixels adjacent to each other; classifying each of theplurality of blocks into one of a plurality of block categories based onthe pixel category of each of the plurality of pixels belonging to eachof the plurality of blocks; and selecting the image output mode bysequentially determining whether or not to select an image output modeas one of the plurality of output modes in accordance with an outputmode order allotted to the plurality of output modes, based on a ratioof a classified number of blocks, which is a number of blocks classifiedinto each block category, to a reference number of blocks, wherein in afirst determination in the output mode order, the reference number ofblocks is a total number of the plurality of blocks of the image, and ina second or subsequent determination in the output mode order, thereference number of blocks is a value that is obtained by subtractingthe classified number of blocks in a preceding determination in theoutput mode order from the reference number of blocks in the precedingdetermination in the output mode order.

According to another aspect of the invention, there is provided acomputer readable recording medium storing a computer program forcausing a computer to perform an image processing method of selecting,based on data of an image that is acquired, an image output mode foroutputting the image from one of a plurality of output modes, the imageprocessing method comprising: classifying each pixel of the image intoone of a plurality of pixel categories based on the image data of eachpixel; dividing the image into a plurality of blocks, the plurality ofblocks each including a plurality of pixels adjacent to each other;classifying each of the plurality of blocks into one of a plurality ofblock categories based on the pixel category of each of the plurality ofpixels belonging to each of the plurality of blocks; and selecting theimage output mode by sequentially determining whether or not to selectan image output mode as one of the plurality of output modes inaccordance with an output mode order allotted to the plurality of outputmodes, based on a ratio of a classified number of blocks, which is anumber of blocks classified into each block category, to a referencenumber of blocks, wherein in a first determination in the output modeorder, the reference number of blocks is a total number of the pluralityof blocks of the image, and in a second or subsequent determination inthe output mode order, the reference number of blocks is a value that isobtained by subtracting the classified number of blocks in a precedingdetermination in the output mode order from the reference number ofblocks in the preceding determination in the output mode order.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a side sectional view showing an image reading apparatusaccording to an illustrative embodiment of the invention;

FIG. 2 is a block diagram showing a configuration of an electroniccircuit of the image reading apparatus shown in FIG. 1;

FIG. 3 (3A, 3B, 3C) is a view illustrating a point to be improved in animage processing method that was suggested by the inventors before theinvention;

FIG. 4 (4A, 4B, 4C) is a view illustrating function effects of an imageprocessing method that is executed by the image reading apparatus shownin FIG. 1;

FIG. 5 (5A, 5B, 5C, 5D, 5E) is another view illustrating the functioneffects of the image processing method;

FIG. 6 (6A, 6B, 6C) is a view illustrating another point to be improvedin an image processing method that was suggested by the inventors beforethe invention;

FIG. 7 (7A, 7B) is a view illustrating function effects of the imageprocessing method that is executed by the image reading apparatus shownin FIG. 1;

FIG. 8 is a flowchart conceptually showing an image processing programthat is executed by a computer of a control circuit unit shown in FIG.2;

FIG. 9 is a flowchart conceptually showing details of S1 of FIG. 8;

FIG. 10 is a flowchart conceptually showing details of S2 of FIG. 8;

FIG. 11 is a flowchart conceptually showing details of S5 of FIG. 8;

FIG. 12 is a flowchart conceptually showing details of S8 of FIG. 8; and

FIG. 13 (13A, 13B) is a flowchart conceptually showing details of S11 ofFIG. 8.

DETAILED DESCRIPTION

Hereinafter, an image reading apparatus 10 according to a specific andexemplary embodiment of the invention will be described in detail withreference to the drawings.

FIG. 1 is a sectional view showing a hardware configuration of the imagereading apparatus 10. The image reading apparatus 10 scans and reads atleast one of an image displayed on an upper surface of a document havingan image, for example, text, photograph and graphic, to be read, i.e., asheet P, and an image displayed on a lower surface of the sheet P, in amain scanning direction, i.e., a direction perpendicular to the sheetsurface in FIG. 1, while conveying the sheet P in a sub-scanningdirection, i.e., a direction parallel to the sheet surface in FIG. 1.Thereby, the image reading apparatus 10 reads the imagetwo-dimensionally and thus generates image data.

The image reading apparatus 10 may be incorporated into or externallymounted to a printer not shown, which prints an image based on the imagedata generated as described above, or a multifunctional machine notshown having a part functioning as a printer.

The image reading apparatus 10 also has an output mode function ofautomatically selecting a mode for outputting an image the on a printingmedium of a printer not shown or a screen of a display not shown, basedon the image data, without user intervention. The mode for outputtingthe image can be selected from a plurality of output modes including amonochrome output mode and a color output mode.

Throughout the description, the term “monochrome output mode” should beinterpreted to include at least the latter of a mode of outputting animage such that the total image has a background color, i.e., the totalimage does not include a colored pixel at all, irrespective of chromaticcolor and achromatic color, and a mode of outputting an image so as toinclude a black pixel. Further, the term “monochrome pixel” should beinterpreted to include at least the latter of a white pixel, i.e., apixel having a background color, and a black pixel. In addition, theterm “monochrome block” should be interpreted to include at least thelatter of a white block, i.e., a block having a background color on thewhole, and a black block having black pixels.

The output mode function is equivalent to a function of classifying ordistinguishing an image (hereinafter, referred to as “target image”),which is indicated by the image data, into one of a plurality of imagecategories including a monochrome image and a color image with respectto types of colors and the number of colors (color characteristics) usedfor the image.

According to the output mode selection function, i.e., image categorydistinguishing function, the following cases can be prevented. First, acase where a processing for outputting a color image, for example,consumption of color toners, is uselessly performed to the image thoughthe read image is actually a monochrome image can be prevented. Second,a case where an image is output as a monochrome image though the readimage is actually a color image, and thus, the original image is not becorrectly reproduced, can be prevented.

As shown in FIG. 1, the image reading apparatus 10 has a sheet feedingopening 14, an image reading unit 16 that optically reads an image onthe sheet P and a sheet discharge opening 18 from an upstream side,i.e., right side in FIG. 1, of a conveyance path 12 of the sheet Ptoward a downstream side, i.e., left side in FIG. 1. The sheet feedingopening 14 is mounted with a sheet feeding tray 20 that can stack andaccommodate therein a plurality of sheets P on standby before they arefed to the sheet feeding opening 14. In the meantime, the sheetdischarge opening 18 is mounted with a sheet discharge tray 22 that canstack and accommodate therein the sheets P discharged from the sheetdischarge opening 18.

The image reading apparatus 10 further has an upper side frame 30 and alower side frame 32, which are spaced with the conveyance path 12 beinginterposed therebetween. The upper side frame 30 and the lower sideframe 32 are connected to each other so that they can be switchedbetween a using position, shown, and a deployed position, not shown, byrotation about one axis line, i.e., an axis line, not shown, extendingin a direction perpendicular to the sheet surface of FIG. 1. A coverswitch 36, as shown in FIG. 2, is provided so as to detect whether theupper side frame 30 is located at the closest position to the lower sideframe 32 or at the most distant position from the lower side frame.

The image reading apparatus 10 is provided at its image reading unit 16with an upper side line sensor 40 and a lower side line sensor 42. Theupper side line sensor 40 is a sensor that is mounted to the upper sideframe 30 and optically reads an image on an upper surface, i.e., frontside, of the sheet P located at a predetermined image reading position.Compared to this, the lower side line sensor 42 is a sensor that ismounted to the lower side frame 32 and optically reads an image on alower surface, i.e., back side, of the sheet P located at the imagereading position.

The upper side line sensor 40 and the lower side line sensor 42 have,respectively, an optic unit , which includes, for example, a lightsource, a lens and a light receiving device, for example, Charge-CoupledDevice, and are closely contacted to the sheet P with contact glasses44, 46 thereof being interposed therebetween.

In the respective line sensors 40, 42, the light sources emit lights,the emitted lights pass through the corresponding contact glasses 44, 46and illuminate the sheet P and the reflected lights from the sheet P arecollected by the lenses and then enter to the light receiving devices.The light receiving device spectrally divides the light entered to thelight receiving device into red component light (R), green componentlight (G) and blue component light (B) and converts the same into a redsignal (R), a green signal (G) and a blue signal (B). The three colorsignals are collectively referred to as RGB signals.

The three color signals are converted into pixel data that indicates anR value, i.e., a red brightness value, a G value, i.e., a greenbrightness value and a B value i.e., a blue brightness value, for eachpixel. The pixel data is binary digital data having 8 bits for each ofRGB values. The respective RGB values indicate the brightness values of256 levels from 0 to 255. The RGB value (0, 0, 0) indicates black andthe RGB value (255, 255, 255) indicates white.

As shown in FIG. 1, the image reading apparatus 10 has, as at least oneconveyance means, a first conveyance roller 50, a second conveyanceroller 52, a third conveyance roller 54 and a sheet discharge roller 56side by side in the sub-scanning direction so as to convey the sheet Palong the conveyance path 12. A first sheet sensor 60, a second sheetsensor 62 and a third sheet sensor 64 shown in FIG. 2, which detectwhether or not the sheet P exists, is provided adjacent to the first,second and third conveyance rollers 50, 52, 54, respectively.

In order to respectively rotate the first, second and third conveyancerollers 50, 52, 54 and the sheet discharge roller 56, the image readingapparatus 10 has a sheet delivery motor 68 common to the first, secondand third conveyance rollers 50, 52, 54 and the sheet discharge roller56. In order to switch the first conveyance roller 50 between a rotatingstate and a stationary state, a first keep solenoid 70 shown in FIG. 2is provided. In order to switch the second conveyance roller S2 betweena rotating state and a stationary state, a second keep solenoid 72 shownin FIG. 2 is provided.

FIG. 2 is a block diagram showing a configuration of an electroniccircuit of the image reading apparatus 10. The image reading apparatus10 has a control circuit unit 80 that controls the whole image readingapparatus, an input/output interface 82 that is electrically connectedto the control circuit unit 80, a motor driving circuit 84 that iselectrically connected to the input/output interface 82, a line sensordriving circuit 86 and a solenoid driving circuit 88.

The motor driving circuit 84 is electrically connected with the sheetdelivery motor 68, the line sensor driving circuit 86 is electricallyconnected with the upper side and lower side line sensors 40, 42 and thesolenoid driving circuit 88 is electrically connected with the first andsecond keep solenoids 70, 72.

The input/output interface 82 is also electrically connected with astart switch 90 that is operated to start or stop the image readingapparatus 10, an error lamp 92 that is turned on so as to warnabnormality of the image reading apparatus 10, the first, second andthird sheet sensors 60, 62, 64 and the cover switch 36, respectively.

The control circuit unit 80 is configured mainly by a computer.Specifically, a Central Processing Unit (CPU) 94 as a processor, a ReadOnly Memory (ROM) 96, a flash ROM 98 and a Random Access Memory RAM 100as memories and a communication interface (I/F) 102 are connected toeach other by a bus 104, thereby configuring the control circuit unit is80. The communication I/F 102 is connected with a Personal Computer (PC)as an external apparatus 110 in a wired or wireless manner.

The ROM 96 is a recording medium for storing in advance a controlparameter that is necessary for controlling the image reading apparatus10, a program, not shown, that is executed by the CPU 94 for conveyancecontrol of the sheet P, and the like. The ROM 96 also stores in advancea plurality of threshold values, a plurality of counters and a pluralityof reference values, which will be described later. The plurality ofreference values includes the reference number of pixels and thereference number of blocks.

The RAM 100 is a recording medium for temporarily storing image dataindicating an image on the upper surface of the sheet P, which is readby the upper side line sensor 40, image data indicating an image on thelower surface of the sheet P, which is read by the lower side linesensor 42, and the like. The image data is temporarily stored in the RAM100 in a page unit.

Referring to FIG. 8, the flash ROM 98 stores an image processing programthat is executed by the CPU 94 so as to select the image output mode asone of the plurality of output modes for each page of the sheets P.

In this illustrative embodiment, a part of the control circuit unit 80,which executes the image processing program, configures an example ofthe “image processing apparatus” of the invention. However, as anotherexample, the “image processing apparatus” of the invention can beincorporated into a printer or a multifunctional machine.

The control circuit unit 80 executes the image processing program,thereby implementing an image processing method that will be describedbelow. The image processing method is an example of the “imageprocessing method” of the invention and the flash ROM 98 that stores animage processing program therein is an example of the “recording medium”of the invention.

In the meantime, the inventor of the invention suggested the followingimage processing method prior to the invention. It is a method ofselecting, based on image data that is a set of a plurality of pixeldata each indicating colors of a plurality of pixels of an image of onepage, an image output mode of outputting the image, which is hereinafterreferred to as “target image”, in one of a plurality of output modes.The plurality of output modes includes a monochrome output mode and acolor output mode. The image processing method is performed for eachpage.

The above suggested image processing method which is hereinafterreferred to as “suggested method”, includes a pixel classificationprocess and an output mode selection process. In the pixelclassification process, the pixels configuring the target image arerespectively classified into one of a plurality of pixel categoriesincluding monochrome and color pixels, based on the corresponding pixeldata. In the output mode selection process, based on the respectivepixel categories of the pixels configuring the target image, the imageoutput mode of the target image is selected as one of the plurality ofoutput modes in accordance with an output mode order that is an orderallotted to the output modes in advance.

Here, the configuration of “selecting one output mode for the targetimage” is equivalent to a configuration of distinguishing, i.e.,classifying the target image as one of a plurality of image categoriesincluding a monochrome image and a color image.

Specifically, the above output mode selection process is implemented asfollows.

In an ith selection, when a ratio RPi of the number of pixels, which isa ratio of the ith classified number of pixels NPi to the referencenumber of pixels, is larger than a threshold value THPi, an ith outputmode of the output mode order is selected as the image output mode.Here, i is an integer of 1 or larger. Here, the ith classified number ofpixels NPi is the number of pixels classified into an ith pixel categoryin the pixel category order among the pixels configuring the targetimage. According to the suggested method, the reference number of pixelsis fixedly set as the total number of pixels included in the targetimage.

However, the inventors found that the suggested method needs to beimproved.

Specifically, according to the suggested method, as described above, theimage output mode of the target image is selected based on the ratio ofthe number of pixels of the target image, which is a ratio of theclassified number of pixels for each image category to the total numberof pixels. For example, when a ratio of the number of color pixels,which is a ratio of the number of color pixels to the total number ofpixels in a target image, is larger than a threshold value, a coloroutput mode is selected for the target image.

Meanwhile, when the target image is actually a color image, the targetimage naturally includes the color pixels. Accordingly, when a thresholdvalue is set so that the ratio of the number of color pixels exceeds thethreshold value, the color output mode is selected for the target imagethat is actually a color image, as expected.

However, even if the target image is actually a monochrome image, if thetarget image includes color pixels and the ratio of the number of colorpixels exceeds the threshold value, the color output mode may beerroneously selected for the target image that is actually a monochromeimage.

For example, in some cases, the colors of the respective pixels of thetarget image that is actually a monochrome image may be separated into aplurality of color components, for example, R, B and G components, whenthe target image is being read by a scanner. In this example, if a pixelto be read is a black pixel, the black pixel will be read so that if theplurality of color components of the black pixel is combined, thecomposed color will be black. However, when a problem of color shiftoccurs, any one of the color components is intensively read with respectto the black pixel, compared to the other color components. As a result,the pixel that is actually the black pixel is erroneously classifiedinto a color pixel.

Like this, when many pixels are erroneously classified as color pixelsand thus the ratio of the number of color pixels resultantly exceeds athreshold value, the color output mode is erroneously selected for thetarget image that is actually the monochrome image. Hereinafter, such aproblem is specifically described with reference to the followingspecific image examples.

In a first image example shown in FIG. 3A, in a sheet P of one page,which is a document on which an image to be read is displayed, andwhich is simply also referred to as “document” in the below, a pluralityof same letters is arranged in a single horizontal row in a narrow area,i.e., an area corresponding to one row. The first two letters of theletters are red and the other letters are black. Compared to this, in asecond image example shown in FIG. 3B, a plurality of same letters, thenumber of which being more than that of the first image example, isarranged in every direction over substantially the entire area of thedocument and all the letters are black.

In FIG. 3C, threshold values are shown which are used to classify thetarget image, i.e., an image on the document, of one page into one of awhite image, which is an image having only background color and is ablank page, a color image, a gray image and a monochrome image, which isan image consisting of a background having a background color and blackpixels.

In all the image examples, as shown in the leftmost side of FIG. 3B, theabove-described color shift occurs when reading a black letter.Accordingly, it is assumed that each black letter, which naturallyconsists of black pixels only, is read to have not only the black pixelsbut also color pixels, i.e., in the example of FIG. 3B, the colors thatare read for each part of the letter are different between both edges ofthe part and a central portion of the part and a stripe pattern isformed as a result of the reading. The total number of black letters ofthe second image example is larger than that of the first image example.Accordingly, in the second image example, the total number of unexpectedcolor pixels, which are pixels that should be originally read as blackpixels, caused due to the color shift is larger than that of the firstimage example.

It is natural that the first image example is classified into the colorimage and the second image example is classified into the monochromeimage. In the first image example, as can be clearly seen from therightmost reading result of FIG. 3A, the ratio of the number of colorpixels is 1.00% satisfies the condition of being equal to or larger thanthe threshold value 1% shown in FIG. 3C. Accordingly, the first imageexample is classified into the color image, as expected.

However, in the second image example, as can be clearly seen from therightmost reading result of FIG. 3B, even though the second imageexample actually has the black letters only, the ratio of the number ofcolor pixels is 1.00%. Therefore, since the ratio of the number of colorpixels of the second image example satisfies a condition of being equalto or larger than the threshold value 1% shown in FIG. 3C, it isunexpectedly classified into the color image.

Like this, according to the suggested method, when the pixels are readas color pixels due to the color shift at the time of reading an image,the target image that should be originally classified into themonochrome image may be erroneously classified into the color image.

In order to prevent the misclassification of the image, the inventorsdeveloped an image processing method including the following processes.

1.Image Division Process

In this process, an image, which is hereinafter referred to as “targetimage”, of one page of the sheet P is divided into a plurality ofblocks, each of which including a plurality of pixels continuous to eachother. Here, the target image is divided into the blocks so that eachblock has a larger size as a resolution of the corresponding image datais higher.

2. Pixel Classification Process

In this process, the respective pixels are sequentially classified intoone of a plurality of pixel categories including white, color, gray andblack pixels, based on the corresponding pixel data, in accordance witha pixel category order allotted to the pixel categories in advance. Inthis illustrative embodiment, the pixel category order is an order ofwhite, color, black and gray pixels.

3. Block Classification Process

In this process, the respective blocks, which are divided from thetarget image, are sequentially classified into one of a plurality ofblock categories including a white block, a color block, a gray blockand a black block, based on the respective pixel categories of theplurality of pixels belonging to each block, in accordance with a blockcategory order allotted to the block categories in advance. In thisillustrative embodiment, the block category order is an order of white,color, gray and black blocks.

Specifically, the block classification process is implemented asfollows.

In a jth classification, for each block, when a ratio RPj of the numberof pixels, which is a ratio of the jth classified number of pixels NPjto the jth reference number of pixels REFPj, is larger than a thresholdvalue THPj, each block is classified into the jth block category in theblock category order. Here, j is an integer of 1 or larger. Here, thejth classified number of pixels NPj is the number of pixels classifiedinto a jth pixel category in the pixel category order.

4. Output Mode Selection Process

In this process, based on the block category of each of the blocks,which are divided from the target image, an image output mode of thetarget image is sequentially selected as one of a plurality of outputmodes including a white output mode, a color output mode, a gray outputmode and a monochrome output mode in accordance with an output modeorder allotted to the output modes in advance. In this illustrativeembodiment, the output mode order is an order of a white output mode,i.e., a blank page mode, a color output mode, a gray output mode and amonochrome output mode.

Specifically, the output mode selection process is implemented asfollows.

In an ith selection, when a ratio RBi of the number of blocks, which isa ratio of the ith classified number of blocks NBi to the ith referencenumber of blocks REFBi, is larger than a threshold value THBi, the ithoutput mode in the output mode order is selected as the image outputmode of the target image. Here, I is an integer of 1 or larger. Here,the ith classified number of blocks NBi is the number of blocks dividedfrom the target image that is classified into an ith block category inthe block category order.

The effects of the image processing method developed as described above,which is hereinafter, referred to as “developed method”, arespecifically described with reference to the first and second imageexamples.

FIG. 4 shows a result that is obtained by implementing the blockclassification process to the first image example shown in FIG. 3A.Specifically, FIG. 4A shows that the target image is divided into aplurality of blocks with respect to the first image example and that forthe first block of the plurality of blocks, the classified number ofpixels in the block and the ratio of the number of pixels in the blockare acquired for each pixel category. In the first block, the red letterexists by at least one letter.

FIG. 4B shows that for the sixth block of the plurality of blocks, theclassified number of pixels in the block and the ratio of the number ofpixels in the block are acquired for each pixel category. In the sixthblock, the black letter exists by about 1.5 letters. FIG. 4C shows thethreshold values that are used to classify the respective blocks intoone of the white block, the color block, the gray block and the blackblock.

As shown at the rightmost side of FIG. 4A, since the ratio of the numberof color pixels is larger than the threshold value in the first block,the first block is classified into the color block. However, since theratios of the numbers of the white pixel, the color pixel and the graypixel respectively satisfies the condition of being the correspondingthreshold values or smaller in the sixth block, the sixth block isclassified into the black block.

A left side of FIG. 5A shows that the target image is divided into aplurality of blocks and a right side of FIG. 5A shows that the blockclassification process is performed for all the blocks, with respect tothe first image example. At the right side of FIG. 5A, a symbol “C”indicates that a block having the symbol attached thereto is classifiedinto the color block, a symbol “B” indicates that a block having thesymbol attached thereto is classified into the black block and a symbol“W” indicates that a block having the symbol attached thereto isclassified into the white block.

As shown in FIG. 5A, in the first image example that is a text image inwhich a plurality of letters is only in the first row of one page, thefirst and second blocks, in which the first and second red letters existrespectively, are classified into color blocks. The other blocksbelonging to the same row and having the black letters are classifiedinto black blocks. The other blocks having the background color isclassified into white blocks.

Regarding the second image example shown in FIG. 3B, in which the imagedata has color pixels generated due to the color shift, a left side ofFIG. 5B shows that the target image is divided into a plurality ofblocks and the right side of FIG. 5B shows that the block classificationprocess is performed for all the blocks.

As shown in FIG. 5B, in the second image example that is a text image inwhich a plurality of letters exists on the whole one page, when it isassumed that the color shift occurred to the image data of all blackletters on the document, since at least one letter exists in each block,the unexpected color pixels exist in all of the blocks. Even when thetotal number of the unexpected color pixels exceeds a threshold value asa whole of the text, a number of the color pixels generated to theletters existing in the respective blocks are small for each of theblocks, compared to the first image example, so that the number of colorpixels for each lock does not exceed the threshold value. Accordingly,all blocks are classified into the black block in the second imageexample.

According to the analysis of the classification results shown in FIGS.5A and 5B, like the first image example, when the object in the targetimage is a text, a part of the letters configuring the text may becolored due to the highlighting. At this time, generally, each coloredletter is colored entirely. Accordingly, when at least one coloredletter exists in a block, the color pixels tend to intensively occur inthe corresponding block. Hence, the number of color pixels tends toincrease in the block in which the color letter exists. Thus, as shownin FIG. 5A, the block in which the color letter exists shows a strongtendency that to be classified into the color block.

In contrast to this, like the second image example, when the object inthe target image is a text, although all the letters configuring thetext are black and are not colored in another color for highlighting,the color shift may occur at the time of reading the image. In general,the color shift shows a strong tendency that it occurs over the entiretext, not locally. Also, the color shift shows a strong tendency that itdoes not occur over the entire letter but partially occurs only at thesurroundings of an outline of each letter.

Hence, the image data, which is obtained by reading the text having thecolor shift, has a possibility that, although the number of unexpectedcolor pixels in a letter is smaller than that in the highlightedletters, since the unexpected color pixels are dispersed over the entiretext, the number of unexpected color pixels in the whole text may beequal to or larger than the number of color pixels corresponding to thehighlighted letters. That is, although the number of unexpected colorpixels is a few from a local standpoint, if the number of letters towhich the color shift occurs is large as a whole, the number ofunexpected color pixels may be equal to or larger than the number ofcolor pixels corresponding to the highlighted letters.

However, from a standpoint of a block unit, since the number of lettersexisting in each block is small, such as one letter, the number ofunexpected color pixels in each block is also small, so that the ratioof the number of color pixels in each block does not exceed thethreshold value.

Meanwhile, in order that the first image example and the second imageexample are correctly classified into the color image and the monochromeimage, respectively, a following measure has been considered. That is, athreshold value that should be compared with the ratio of the number ofcolor pixels of the whole document is set to be slightly higher, so thata result of classifying a document in which the image data has theexpected color pixels and a result of classifying a document in whichthe image data has the unexpected color pixels are different.

However, by the above measure, in some cases, even the ratio of thenumber of color pixels of the document in which the image data has theexpected color pixels may not exceed the threshold value set to beslightly higher, so that the document can not be classified into thecolor image.

Regarding the above problem, according to the developed method, theratio of the number of color pixels is calculated in a block unit, sothat the ratio of the number of color pixels is suppressed from beingincreased due to the color shift. As a result, the block in which theimage data has the expected color pixels can be classified into thecolor block and a block in which the image data has the unexpected colorpixels can be classified into the monochrome block, without setting thethreshold value to be slightly higher.

That is, the illustrative embodiment pays attention to the fact that theunexpected color pixels generated due to the color shift have a tendencyto disperse in a low density on the image data, i.e., document. In theillustrative embodiment, the target image is first classified in a pixelunit and is then classified in a block unit including a plurality ofpixels, based on the first classification. Accordingly, it is possibleto distinguish a case where the image data has the expected color pixelsand a case where the image has the unexpected color pixels.

FIG. 5C shows a result that is obtained by performing the output modeselection process to the first image example and FIG. 5D shows a resultthat is obtained by performing the output mode selection process to thesecond image example. FIG. 5E shows threshold values that are used toclassify the target image into one of white, color, gray and monochromeimages.

As shown in FIG. 5C, in the first image example, the ratio of the numberof blocks of the color blocks satisfies the condition of being thethreshold value or larger, so that the first image example is classifiedinto the color image, as expected. Compared to this, as shown in FIG.5D, in the second image example, since the ratios of the number ofblocks of the white, color and gray blocks are smaller than thecorresponding threshold values, respectively, the target image isclassified into the monochrome image, as expected.

However, the inventors found that the suggested method needs to beimproved.

Specifically, considering a case where one object is displayed in colorin a sheet P of one page which is hereinafter referred to as “document”,there are cases where a same object is displayed in large and smallsizes on the document. The object displayed in a small size is obtained,for example, by reducing the object displayed in a large size, and theobject displayed in a large size is obtained, for example, by enlargingthe object displayed in a small size.

Ideally, it is natural that both the documents are classified into thecolor images and the color output mode is selected. However, since thedocument on which the object is displayed in a small size has thesmaller number of color pixels occupying the whole document, compared tothe document on which the object is displayed in a large size, the ratioof the number of color pixels is smaller.

Accordingly, regarding the document on which the object is displayed ina small size, the ratio of the number of color pixels is smaller thanthat of the document on which the object is largely displayed and thusdoes not exceed the threshold value and the corresponding document isclassified into an image, for example, monochrome image, rather than thecolor image, so that the monochrome mode may be selected, even though itis natural that the document is to be classified into the color image.

That is, according to the suggested method, since the ratio of thenumber of color pixels has a strong tendency to depend on a size of apart of the document on which the color object is not displayed, i.e., amargin or a background part, the color characteristics of the image maynot be correctly distinguished, so that an erroneous image output modemay be selected. Hereinafter, such a problem is specifically describedwith reference to the following specific image examples.

In a third image example shown in FIG. 6A, a color object is displayedin a slightly large area of the document. Compared to this, in a fourthimage example shown in FIG. 6B, a color object that is a reduced objectof the third image example is displayed in a small area of the document.FIG. 6C shows threshold values that are used to classify the targetimage of one page, i.e., image on the document, into one of white,color, gray and monochrome images.

It is natural that both the third and fourth image examples areclassified into the color image. In the third image example, as can beclearly seen from the reading result at the rightmost of FIG. 6A, theratio of the number of color pixels is 15%, which is larger than thethreshold value of 2% shown in FIG. 6C. Accordingly, the third imageexample is classified into the color image, as expected.

However, in the fourth image example, as can clearly seen from thereading result at the rightmost of FIG. 6B, the ratio of the number ofcolor pixels is 1.5%. Therefore, since the ratio of the number of colorpixels in the fourth image example is smaller than the threshold valueof 2% shown in FIG. 6C, the fourth image example is unexpectedlyclassified into the monochrome image.

Like this, according to the suggested method, when the area of the parton which the color object is not displayed, i.e., the area of the marginor background part, is large, the color characteristics of the image maynot be resultantly distinguished correctly, so that an erroneous imageoutput mode may be selected. This should be improved.

Regarding this, the inventors improved the suggested method so that thereference number of pixels is not fixed as the total number of pixelsbut is varied, i.e., is decreased as the pixel classification process isperformed in the pixel category order. According to this improvedsuggested method, which is hereinafter referred to as “improved method”,the output mode selection process is sequentially implemented inaccordance with the pixel category order, as follows.

(1) In a first selection, when a ratio of the number of pixels RP1,which is a ratio of the first classified number of pixels NP1 to thefirst reference number of pixels REFP1, is larger than a threshold valueTHP1, a first output mode in the output mode order is selected as theimage output mode of the target image. Here, the first number of pixelsNP1 is the number of pixels configuring the target image, which isclassified into a first pixel category in the pixel category order.Further, the “the first reference number of pixels REFP1” is the totalnumber of pixels of the target image.

(2) In an ith selection, when a ratio RPi of the number of pixels thatis a ratio of the ith classified number of pixels NPi to the ithreference number of pixels REFPi is larger than a threshold value THPi,an ith output mode in the output mode order is selected as the imageoutput mode. Here, i is an integer of 2 or larger. Here, the ithclassified number of pixels NPi is the number of pixels configuring thetarget image, which is classified into an ith pixel category of thepixel category order.

Here, the “the ith reference number of pixels REFPi” is a value that isobtained by subtracting the (i-1)th classified number of pixels NPi-1from the (i-1)th reference number of pixels REFPi-1.

For example, the second reference number of pixels is the same as avalue that is obtained by subtracting the number of white pixels fromthe total number of pixels. The second reference number of pixels isapproximate to the number of pixels configuring a part, for example,object part, except for a white part, typically, margin part, of thewhole document. Accordingly, the second ratio of the number of pixels iscalculated without being influenced by the other white part as much aspossible. As a result, a ratio of the object in the document to thedocument size influences the second ratio of the number of pixels only alittle.

The effects of the improved method are specifically described withreference to the third and fourth image examples.

In the third image example, as shown in FIG. 7A, the ratio of the numberof color pixels is 50%, which is larger than the threshold value of 2%shown in FIG. 6C. Hence, the third image example is classified into thecolor image, as expected. Also, in the fourth image example, as shown inFIG. 7B, the ratio of the number of color pixels is also 50%.Accordingly, since the fourth image example has the ratio of the numberof color pixels that is larger than the threshold value of 2% shown inFIG. 6C, the fourth image example is also classified into the colorimage, as expected. Thereby, the misclassification of the target image,which is caused due to the size of the object in the document, isprevented.

The inventors paid attention to the fact that a configuration in whichthe target image is configured by the plurality of pixels and aconfiguration in which the target image is configured by the pluralityof blocks are common in that the target image is configured by aplurality of partial images. Based on this, the inventors applied theabove technical spirit of setting the reference number of pixels as avariable value to the “reference number of blocks” in the output modeselection process of the developed method. That is, the inventors setthe reference number of blocks as a variable value in the output modeselection process. Specifically, the output mode selection process ofthe image processing method of this illustrative embodiment isimplemented as follows.

(1) First Selection

In the first selection, when a ratio RB1 of the number of blocks, whichis a ratio of the first classified number of blocks NB1 to the firstreference number of blocks REFB1, is larger than a threshold value THB1,a first output mode in the output mode order is selected as the imageoutput mode of the target image. In this illustrative embodiment, thefirst output mode is a white output mode. Here, the first classifiednumber of blocks NB1 is, of the blocks divided from the target image,the number of blocks classified into a first block category in the blockcategory order. In this illustrative embodiment, the first blockcategory is a white block. Further, the “first reference number ofblocks REFB1” is a total number of blocks of the target image.

(2) ith (i: integer of 2 or greater) Selection, i.e., second orsubsequent selection

In an ith selection, when a ratio RBi of the number of blocks, which isa ratio of the ith classified number of blocks NBi to the ith referencenumber of blocks REFBi, is larger than a threshold value THBi, an ithoutput mode of the output mode order is selected as the image outputmode of the target image. Here, I is an integer of 2 or larger. Here,the ith classified number of blocks NBi is the number of blocks, of theblocks divided from the target image, classified into an ith blockcategory of the block category order. Further, the “the ith referencenumber of blocks REFBi” is a value that is obtained by subtracting the(i-1)th classified number of blocks NBi-1 from the (i-1)th referencenumber of blocks REFBi-1.

That is, according to the illustrative embodiment, the reference numberof blocks is not constant in all of the selections. That is, thereference number of blocks is the same as the total number of blocks inthe first selection. However, in each of the second or subsequentselections, the reference number of blocks is the same as a value thatis obtained by subtracting the classified number of blocks in apreceding selection from the reference number of blocks in the precedingselection and decreases as the number of the selection increases.Accordingly, the reference number of blocks that is used in each of thesecond and subsequent selection is defined so that the number of blocks,i.e., used blocks, noted in the selection of previous time is excludedtherefrom. That is, the reference number of blocks that is used in eachof the second and subsequent selection is defined so that the classifiednumber of blocks of a previous selection is excluded therefrom. Hence,the ratio of the number of pixels calculated at the selection of thistime is prevented from becoming an unexpectedly small value due to thereference number of blocks of this time including the number of usedblocks, which exist before the selection of this time, being increased,and the unexpected image output mode being erroneously selected for theimage is prevented.

Also, the inventors paid attention to the fact that the configuration inwhich the target image is configured by the plurality of pixels and theconfiguration in which each block is configured by the plurality ofpixels are common in that the image area is configured by a plurality ofpixels. Based on this, the inventors applied the above technical spiritof setting the reference number of pixels as a variable value to the“reference number of pixels” in the block classification process of thedeveloped method. That is, the inventors set the reference number ofpixels as a variable value in the block classification process.Specifically, the block classification process of the image processingmethod of this illustrative embodiment is implemented as follows.

(1) First Classification

In the first classification, when a ratio RP1 of the number of pixelsthat is a ratio of the first classified number of pixels NP1 to thefirst reference number of pixels REFP1 is larger than a threshold valueTHP1, each block is classified into a first block category, in thisillustrative embodiment, white block, in the block category order. Here,the first classified number of pixels NP1 is the number of pixels, ofthe pixels, i.e., the extracted pixels, belonging to each block,classified into a first pixel category, in this illustrative embodiment,white pixel, of the pixel category order. Further, the “first referencenumber of pixels REFP1” is the same as the total number of pixelsbelonging to each block.

(2) jth (j: integer of 2 or greater) Classification, i.e., second orsubsequent classifications

In a jth classification, when a ratio RPj of the number of pixels thatis a ratio of the jth classified number of pixels NPj to the jthreference number of pixels REFPj is larger than a threshold value THPj,each block is classified into a jth block category of the block categoryorder. Here, the jth classified number of pixels NPj is the number ofpixels, of the pixels belonging to each block, classified into a jthpixel category in the pixel category order. Here, the “the jth referencenumber of pixels REFPj” is a value that is obtained by subtracting the(j-1)th classified number of pixels NPj-1 from the (j-1)th referencenumber of pixels REFPj-1.

Adding a remark, in this illustrative embodiment, the above technicalspirit of setting the reference number of pixels as a variable value isapplied to the reference number of pixels in the block selection processand the reference number of blocks in the output mode selection process.However, the invention can be implemented by applying the technicalspirit to at least the reference number of blocks. Also, the abovetechnical spirit of setting the reference number of pixels as a variablevalue may be applied to at least the reference number of pixels.

In order to implement the image processing method of this illustrativeembodiment, which has been schematically described, an image processingprogram shown in FIG. 8 is executed.

When the image processing program is executed, in S1, an imagehereinafter referred to as “target image”, which is displayed by theimage data of one page stored in the RAM 100, is divided into aplurality of blocks including a plurality of pixels adjacent to eachother. The details of S1 are conceptually shown in a flowchart of FIG.9, as an image division routine.

In the image division routine, in S31, a resolution R of the image datathat displays the target image is read out from the RAM 100. The dataindicating the resolution R is included in or associated with this imagedata.

Then, in S32 to S36, a target size of each of the blocks into which thetarget image should be divided is set to be larger as the readresolution R is higher.

Specifically, in S32, it is determined whether the resolution R is theminimum threshold value R0, for example, 300 [dpi], or lower. Thisdetermination corresponds to a determination of determining whether asize of the target image is A4 size or not. This time, when it isassumed that the resolution R is the threshold value R0 or lower, atarget size of one block is set in S33 so that a block width W is W0,for example, 16 [pixel], and a block height H is H0, for example, 16[pixel].

On the other hand, when the resolution R is higher than the thresholdvalue R0, it is determined in S34 whether the resolution R is athreshold value R1 or lower, which is larger than the threshold valueR0. This determination corresponds to a determination of determiningwhether a size of the target image is B size or not, for example. Thistime, when it is assumed that the resolution R is the threshold value R1or lower, a target size of one block is set in S35 so that the blockwidth W is W1 (>W0) and the block height H is H1 (>H0).

On the other hand, when the resolution R is higher than the thresholdvalue R1, a target size of one block is set in S36 so that the blockwidth W is W2 (>W1) and the block height H is H2 (>H1).

In any case, based on the image data, i.e., original image data, of thetarget image, the target image is divided into a plurality of blocks inS37 so that each block has a target size that is increased as theresolution R is increased. Thereby, a set of a plurality of block dataindicating the blocks is generated as image data after the division.After the division, the image data is preserved in the RAM 100. As aresult, the execution of the image division routine is ended and theprocess returns to S2 of FIG. 8.

In S2, a thinning of the pixel is performed. The details of S2 areconceptually shown in a flowchart of FIG. 10, as a pixel thinningroutine.

In the pixel thinning process, in S51, the resolution R of the imagedata indicating the target image is read out from the RAM 100. Then, inS52 to S59, a thinning ratio of the pixel is set so that it is decreasedas the resolution R is higher.

The term “thinning ratio” means a ratio of the number of pixels of thetarget image after the thinning to the number of pixels of the targetimage before the thinning When the thinning ratio is expressed by “1/k”,it means that one pixel of k pixels is extracted as an effective pixeland (k-1) pixels of k pixels are skipped. Here, k is an integer of 1 orlarger. The lower the thinning ratio, the number of skipped pixels of aplurality of pixels originally belonging to the target image isincreased.

Specifically, in S52, it is determined whether the resolution R is theminimum threshold value R10 or lower. This time, when it is assumed thatthe resolution R is the threshold value R10 or lower, the thinning ratiois set 1/1 in S53. The thinning ratio of 1/1 means that the thinning isnot performed for the target image and all pixels configuring the targetimage are extracted as the effective pixels.

On the other hand, when the resolution R is higher than the thresholdvalue R10, it is determined in S54 whether the resolution R is athreshold value R11 or lower, which is larger than the threshold valueR10. This time, when it is assumed that the resolution R is thethreshold value R11 or lower, the thinning ratio is set 1/2 in S55. Thethinning ratio of 1/2 means that one pixel to two pixels with respect tothe pixels configuring the target image is extracted and the other onepixel is skipped.

On the other hand, when the resolution R is higher than the thresholdvalue R11, it is determined in S56 whether the resolution R is athreshold value R12 or lower, which is larger than the threshold valueR11. This time, when it is assumed that the resolution R is thethreshold value R12 or lower, the thinning ratio is set 1/4 in S57. Thethinning ratio of 1/4 means that one pixel to four pixels with respectto the pixels configuring the target image is extracted and the otherthree pixels are skipped.

On the other hand, when the resolution R is higher than the thresholdvalue R12, the thinning ratio is set 1/8 in S58. The thinning ratio of1/8 means that one pixel to eight pixels with respect to the pixelsconfiguring the target image is extracted and the other seven pixels areskipped.

In any case, in S59, the plurality of pixel data configuring the imagedata of the target image is thinned with the thinning ratio, which isdecreased as the resolution R becomes higher, so that a plurality ofeffective pixel data is extracted. The effective image data is preservedin the RAM 100. Thereby, the target image is divided into a plurality ofblocks so that each block has only the effective pixels. As a result,the execution of the pixel thinning routine is ended and the processreturns to S3 of FIG. 8.

In S3, when it is supposed that the blocks belonging to the target imageare laid out in a row, a leading block is selected as a target block.Subsequently, in S4, when it is supposed that a plurality of pixelsbelonging to the target block is laid out in a row, a leading pixel isselected as a target pixel. Here, although the pixels are the effectivepixels extracted by the thinning of S2, each of them are simply referredto as “pixel”.

Then, in S5, the target pixel is classified into one of a white pixel, acolor pixel, a gray pixel and a black pixel. The details of S5 areconceptually shown in a flowchart of FIG. 11, as a pixel classificationroutine.

In the pixel classification routine, in S101, regarding the targetpixel, it is determined whether all the R, G and B values are a whitethreshold value, for example, “240”, or larger, which is a thresholdvalue for classifying the target pixel as a white pixel. Regarding thetarget pixel, when it is assumed that all the RGB values are the whitethreshold value or larger, the target pixel is classified into a whitepixel. Then, in S102, regarding the target pixel, a block white pixelcounter for counting the number of white pixels existing in the targetblock is incremented by 1. As a result, one execution of the pixelclassification routine is ended and the process returns to S6 of FIG. 8.

On the other hand, regarding the target pixel, when it is assumed thatat least one of the RGB values is smaller than the white thresholdvalue, a brightness difference |R−G| between red and green, i.e., anabsolute value of a difference between R and G brightness values, abrightness difference |G−B| between green and blue, i.e., an absolutevalue of a difference between G and B brightness values, and abrightness difference |B−R| between blue and red, i.e., an absolutevalue of a difference between B and R brightness values, are calculatedfor the target pixel in S103.

Subsequently, regarding the target pixel, it is determined in S104whether at least one of the three brightness values is a color thresholdvalue, for example, “10”, which is a threshold value for classifying thetarget pixel as a color pixel or larger,. Regarding the target pixel,when it is assumed that at least one of the three brightness differencesis the color threshold value or larger, the target pixel is classifiedinto a color pixel. Then, in S105, regarding the target block, a blockcolor pixel counter for counting the number of color pixels existing inthe target block is incremented by 1. As a result, one execution of thepixel classification routine is ended and the process returns to S6 ofFIG. 8.

On the other hand, regarding the target pixel, when it is assumed thatany brightness difference is smaller than the color threshold value, itis determined for the target pixel whether all the R, G and B values area black threshold value, for example, “50”, or smaller, which is athreshold value for classifying the target pixel as a black pixel.Regarding the target pixel, when it is assumed that all the RGB valuesare the black threshold value or smaller, the target pixel is classifiedinto a black pixel. Then, in S107, regarding the target block, a blackpixel counter in a block for counting the number of black pixelsexisting in the target block is incremented by 1. As a result, oneexecution of the pixel classification routine is ended and the processreturns to S6 of FIG. 8.

On the other hand, regarding the target pixel, when it is assumed thatat least one of the RGB values is larger than the black threshold value,the target pixel is classified into a gray pixel. Then, in S108,regarding the target block, a block gray pixel counter for counting thenumber of gray pixels existing in the target block is incremented by 1.As a result, one execution of the pixel classification routine is endedand the process returns to S6 of FIG. 8.

In S6 of FIG. 8, it is determined whether the execution of S5 is endedfor all pixels belonging to the target block. When it is assumed thatthe execution is not ended yet, the target pixel is moved to a pixelfollowing the current pixel in S7. Subsequently, returning to S5, the S5is executed for a next pixel. On the other hand, when it is assumed thatthe execution of S5 is ended for all pixels belonging to the targetblock, the process proceeds to S8.

In S8, the target block is classified into one of white, color, gray andblack blocks. The details of S8 are conceptually shown in a flowchart ofFIG. 12, as a block classification routine.

In the block classification routine, in S201, regarding the targetblock, the values of the four pixel counters are summed up, so that atotal sum of the number of white pixels, i.e., a value of the blockwhite pixel counter), the number of color pixels, i.e., a value of theblock color pixel counter, the number of black pixels, i.e., a value ofthe black pixel counter in a block and the number of gray pixels, i.e.,a value of the block gray pixel counter, which belong to the targetblock, is calculated as the total number of pixels of the target block.

Subsequently, in S202, regarding the target block, a ratio of the numberof white pixels is calculated by dividing the number of white pixels bythe first reference number of pixels. The first reference number ofpixels is the calculated total number of pixels.

Then, in S203, it is determined whether the calculated ratio of thenumber of white pixels is larger than a threshold value for white blockfor classifying the target block as a white block. Regarding the targetblock, when it is assumed that the ratio of the number of white pixelsis higher than the threshold value for white block, the target block isclassified into a white block. Then, in S204, regarding the targetimage, a white block counter for counting the number of white blocksexisting in the target image is incremented by 1. As a result, oneexecution of the block classification routine is ended and the processreturns to S9 of FIG. 8.

On the other hand, regarding the target block, when it is assumed thatthe ratio of the number of white pixels is the threshold value for whiteblock or lower, a ratio of the number of color pixels is calculated bydividing the number of color pixels by the second reference number ofpixels with respect to the target block, in S205. The second referencenumber of pixels is a value that is obtained by subtracting the numberof white pixels from the first reference number of pixels, i.e., totalnumber of pixels.

Then, in S206, it is determined whether the calculated ratio of thenumber of color pixels is higher than a threshold value for color blockfor classifying the target block as a color block. Regarding the targetblock, when it is assumed that the ratio of the number of color pixelsis higher than the threshold value for color block, the target block isclassified into a color block. Then, in S207, regarding the targetimage, a color block counter for counting the number of color blocksexisting in the target image is incremented by 1. As a result, oneexecution of the block classification routine is ended and the processreturns to S9 of FIG. 8.

On the other hand, regarding the target block, when it is assumed thatthe ratio of the number of color pixels is the threshold value for colorblock or lower, a ratio of the number of gray pixels is calculated bydividing the number of gray pixels by the third reference number ofpixels with respect to the target block, in S208. The third referencenumber of pixels is a value that is obtained by subtracting the numberof color pixels from the second reference number of pixels, which is avalue that is obtained by subtracting the number of white pixels and thenumber of color pixels from the first reference number of pixels, i.e.,total number of pixels.

Then, in S209, it is determined whether the calculated ratio of thenumber of gray pixels is higher than a threshold value for gray blockfor classifying the target block as a gray block. Regarding the targetblock, when it is assumed that the ratio of the number of gray pixels ishigher than a threshold value for gray block, the target block isclassified into a gray block. Then, in S210, regarding the target image,a gray block counter for counting the number of gray blocks existing inthe target image is incremented by 1. As a result, one execution of theblock classification routine is ended and the process returns to S9 ofFIG. 8.

On the other hand, regarding the target block, when it is assumed thatthe ratio of the number of gray pixels is the threshold value for grayblock or lower, the target block is classified into a black block. Then,in S211, regarding the target image, a black block counter for countingthe number of black blocks existing in the target image is incrementedby 1. As a result, one execution of the block classification routine isended and the process returns to S9 of FIG. 8.

In S9 of FIG. 8, it is determined whether the execution of S8 is endedfor all blocks belonging to the target image. This time, when it isassumed that the execution has not ended yet, the target block is movedto a block following the current block, in S10. Subsequently, theprocess returns to S4 and S4 is executed for a next block. On the otherhand, when it is assumed that the execution of S8 is ended for allblocks belonging to the target image, the process proceeds to S11.

In S11, the target image is classified into one of a white image, i.e.,a blank page, a color image, a gray image and a monochrome image. Thedetails of S11 are conceptually shown in a flowchart of FIG. 13 (13A,13B), as an image classification routine.

In the image classification routine, in S301, regarding the targetimage, the values of the four block counters are summed up, so that atotal sum of the number of white blocks, i.e., the value of the whiteblock counter, the number of color blocks, i.e., the value of the colorblock counter, the number of black blocks, i.e., the value of the blackblock counter and the number of gray blocks, i.e., the value of the grayblock counter, which belong to the target image, is calculated as thetotal number of blocks for the target image.

Subsequently, in S302, regarding the target image, a ratio of the numberof white blocks is calculated by dividing the number of white blocks bythe first reference number of blocks. The first reference number ofblocks is the calculated total number of blocks.

Then, in S303, it is determined whether the calculated ratio of thenumber of white blocks is higher than a threshold value for blank pagefor classifying the target image as a blank page, i.e., a page having abackground color in the entire page. Regarding the target image, when itis assumed that the ratio of the number of white blocks is higher thanthe threshold value for blank page, the target image is classified intoa blank page, in S304.

Then, in S305, the image data indicating the target image is convertedinto monochrome. Specifically, regarding all the pixels, the image datais converted so that each of the RGB values indicates “255.” Theinformation indicating that the target image is a blank page is added toa header of the converted image data, which is then stored in the RAM100. As a result, one execution of the image classification routine isended and the process returns to S12 of FIG. 8.

On the other hand, when it is assumed that the ratio of the number ofwhite blocks is the threshold value for blank page or lower, a ratio ofthe number of color blocks is calculated by dividing the number of colorblocks by the second reference number of blocks with respect to thetarget image, in S306. The second reference number of blocks is a valuethat is obtained by subtracting the number of white blocks from thefirst reference number of blocks, i.e., total number of blocks.

Then, in S307, it is determined whether the calculated ratio of thenumber of color blocks is higher than a threshold value for color imagefor classifying the target image as a color image. Regarding the targetimage, when it is assumed that the ratio of the number of color blocksis higher than the threshold value for color image, the target image isclassified into a color image in S308. The information indicating thatthe target image is a color image is added to a header of the image dataof the target image, which is then stored in the RAM 100. As a result,one execution of the image classification routine is ended and theprocess returns to S12 of FIG. 8.

On the other hand, regarding the target image, when it is assumed thatthe ratio of the number of color blocks is the threshold value for colorimage or lower, a ratio of the number of gray blocks is calculated bydividing the number of gray blocks by the third reference number ofblocks with respect to the target image, in S309. The third referencenumber of blocks is the same as a value that is obtained by subtractingthe number of color blocks from the second reference number of blocks,which is the same as a value that is obtained by subtracting the numberof white blocks and the number of color blocks from the first referencenumber of blocks, i.e., total number of blocks.

Then, in S310, it is determined whether the calculated ratio of thenumber of gray blocks is higher than a threshold value for gray imagefor classifying the target image as a gray image. Regarding the targetimage, when it is assumed that the ratio of the number of gray blocks ishigher than the threshold value for gray image, the target image isclassified into a gray image in S311.

Then, in S312, the image data indicating the target image is convertedinto gray. Specifically, for example, regarding all the pixels, based onthe respective RGB values, gradation conversion of a predeterminednumber of gradations, for example, 256 gradations, is performed, so thatthe image data before conversion is converted into image data indicatinga gradation image. The information indicating that the target image is agray image is added to a header of the converted image data, which isthen stored in the RAM 100. As a result, one execution of the imageclassification routine is ended and the process returns to S12 of FIG.8.

On the other hand, regarding the target image, when it is assumed that aratio of the number of gray blocks is the threshold value for gray imageor lower, the target image is classified into a monochrome image inS313.

Then, in S314, the image data indicating the target image is convertedinto monochrome. Specifically, regarding all the pixels, average valuesof the R values, the B values and the G values are calculated, and basedon the average values, the image data before conversion is convertedinto image data indicating binary. The information indicating that thetarget image is a monochrome image is added to a header of the convertedimage data, which is then stored in the RAM 100. As a result, oneexecution of the image classification routine is ended and the processreturns to S12 of FIG. 8.

In S12, the image data that is processed with respect to the targetimage is output from the RAM 100 to the PC 110 through the communicationI/F 102. As a result, the execution of the image processing program isended. The PC 110 having received the image data prints or displays thetarget image, based on the received image data and the output mode addedto the header of the image data, as required.

Additionally, the invention can also be implemented by variouslymodifying the above-described illustrative embodiment.

For example, in the above-described illustrative embodiment, the targetsize of the block into which the target image should be divided is setbased on the resolution of the image data of the target image. However,the target size may be set based on other parameters.

For example, when the target image is a text, the target size of theblock may be set so that at least one letter exists in each block, basedon font data of a letter, space data between the letters, and the like.

Also, in the above-described illustrative embodiment, when the targetimage is classified into a blank page, the image data before conversionis converted into the image data indicating that the entire image has abackground color, i.e., white. However, the target image, i.e., theimage data corresponding to the current page may be deleted from theimage processing target, for example, data that should be transmitted tothe PC 110, without performing the data conversion. That is, since thetarget image of this time is a white sheet, it is not necessary to printor display the target image thereafter.

Also, in the above-described illustrative embodiment, the pixelclassification, i.e., identification of the color characteristics of thepixel, process is performed in the order of white pixel determination,color pixel determination and black/gray pixel determination, the blockclassification, i.e., identification of the color characteristics of theblock, process is performed in the order of white block determination,color block determination and gray/black block determination and theimage classification, i.e., identification of the color characteristicsof the image or document of one page, process is performed in the orderof blank page determination, color image determination and gray/blackimage determination. However, the determination order may be changed ineach classification process.

Also, there is no limitation to the determination order among the threetypes of classification processes. Accordingly, the determination orderin each classification process may be independently determined.

Also, in the above-described illustrative embodiment, the image that isread by the image reading apparatus 10 is processed by the CPU 94 of theimage reading apparatus 10. However, the processing unit is not limitedto the CPU 94. For example, the processing may be performed by anexternal apparatus such as PC connected to the image reading apparatus10 through the communication I/F 102. In this case, the PC acquires thedata of the image that is read from the sheet P by the image readingapparatus 10, and performs the necessary image processing to the data.

Also, in the above-described illustrative embodiment, as an example ofthe image processing apparatus of the invention, the image readingapparatus 10, which is a so-called sheet feed scan type of reading animage while conveying the sheet P, is adopted. However, the imageprocessing apparatus of the invention is not limited thereto and a flatbed-type scanner, a facsimile apparatus, a multifunctional machine andthe like may also be adopted.

Also, in the above-described illustrative embodiment, the image outputmode of outputting the image, based on the image data of correspondingone page, is selected. However, the invention is not limited thereto.For example, an image output mode of outputting a plurality of images,based on image data of a plurality of corresponding pages, may becollectively selected commonly to the images.

Also, in the above-described illustrative embodiment, each pixel isclassified into one of the plurality of pixel categories including thewhite pixel, the color pixel, the black pixel and the gray pixel.However, the invention is not limited thereto. For example, each pixelmay be classified into one of a predetermined number of pixelcategories, the predetermined number being smaller than a number of thepixel categories of the above-described illustrative embodiment. Forexample, each pixel may be classified into one of the pixel categoriesincluding only the white pixel and the black pixel. Also, the colorpixel may be segmented into R, G and B pixels, so that the color pixelmay be classified into one of the R, G and B pixels.

Although the illustrative embodiments of the invention have beendescribed with reference to the drawings, the invention is not limitedthereto. The illustrative embodiments of the invention are exemplary andcan be variously modified and improved, based on the knowledge of oneskilled in the art.

According to the illustrative embodiments, regarding the target image,the classification is first performed in a pixel unit. Then, based onthe classification result, the classification is performed in a blockunit including a plurality of pixels. Finally, the target image isclassified. Accordingly, a possibility in which an image, which does notactually have a color pixel, will be misclassified into a color imagedue to the color shift at the time of reading the image, is reduced.

Also, according to the illustrative embodiments, since the referencenumber of blocks, which is used in the image output mode determinationof each of the second or subsequent determination, is a variable valuethat is set so as not to include the number of blocks focused in thepreceding determination, i.e., the classified number of blocks in thepreceding determination, the reference number of blocks is smaller thana fixed value that is set to include the classified number of blocks inthe preceding determination. As a result, the ratio that is calculatedin the determination does not become an unexpected small value, so thata possibility that an unexpected image output mode will be erroneouslyselected for the target image is reduced.

What is claimed is:
 1. An image processing apparatus that selects, basedon data of an image that is acquired, an image output mode foroutputting the image from one of a plurality of output modes, the imageprocessing apparatus comprising: a pixel classification unit thatclassifies each pixel of the image into one of a plurality of pixelcategories based on the image data of each pixel; an image division unitthat divides the image into a plurality of blocks, the plurality ofblocks each including a plurality of pixels adjacent to each other; ablock classification unit that classifies each of the plurality ofblocks into one of a plurality of block categories based on the pixelcategory of each of the plurality of pixels belonging to each of theplurality of blocks; and an output mode selection unit that selects theimage output mode by sequentially determining, in accordance with anoutput mode order allotted to the plurality of output modes, whether ornot to select one of the plurality of output modes as the image outputmode, based on a ratio of a classified number of blocks, which is anumber of blocks classified into each block category, to a referencenumber of blocks, wherein the output mode selection unit selects theimage output mode after the block classification unit has classifiedeach of the plurality of blocks into one of the plurality of blockcategories, wherein the plurality of output modes includes at least afirst output mode and a second output mode, the first output mode beingprior to the second output mode in the output mode order, wherein, in afirst determination in the output mode order, the output mode selectionunit determines whether or not to select the first output mode as theimage output mode based on a ratio of a first classified number ofblocks, which is a number of blocks classified into a first blockcategory, to a first reference number of blocks, which is a total numberof the plurality of blocks of the image, and wherein, in a seconddetermination in the output mode order, the output mode selection unitdetermines whether or not to select the second output mode as the imageoutput mode based on a ratio of a second classified number of blocks,which is a number of blocks classified into a second block category, toa second reference number of blocks, which is a value obtained bysubtracting the first classified number of blocks from the firstreference number of blocks.
 2. The image processing apparatus accordingto claim 1, wherein the block classification unit classifies each blockinto one of the plurality of block categories by sequentiallydetermining whether or not to classify each block into one of theplurality of blocks in accordance with a block category order allottedto the plurality of block categories, based on a ratio of a classifiednumber of pixels, which is a number of pixels classified into each pixelcategory, to a reference number of pixels, and wherein in a firstdetermination in the block category order by the block classificationunit, the reference number of pixels is a total number of pixels of theimage, and in a second or subsequent determination in the block categoryorder by the block classification unit, the reference number of blocksis a value that is obtained by subtracting the classified number ofpixels in a preceding determination in the block category order by theblock classification unit from the reference number of pixels in thepreceding determination in the block category order by the blockclassification unit.
 3. The image processing apparatus according toclaim 1, wherein the image division unit divides the image into theplurality of blocks so that each block has a larger size as a resolutionof the image data is higher.
 4. The image processing apparatus accordingto claim 1, further comprising a thinning unit that thins the pluralityof pixels with a thinning ratio that is decreased as a resolution of theimage data increases, and thereby extracts a plurality of effectivepixels, which are pixels to be classified by the pixel classificationunit, from the plurality of pixels.
 5. An image processing method forselecting, based on data of an image that is acquired, an image outputmode for outputting the image from one of a plurality of output modes,the image processing method comprising: classifying each pixel of theimage into one of a plurality of pixel categories based on the imagedata of each pixel; dividing the image into a plurality of blocks, theplurality of blocks each including a plurality of pixels adjacent toeach other; classifying each of the plurality of blocks into one of aplurality of block categories based on the pixel category of each of theplurality of pixels belonging to each of the plurality of blocks; andselecting the image output mode by sequentially determining, inaccordance with an output mode order allotted to the plurality of outputmodes, whether or not to select one of the plurality of output modes asthe image output mode, based on a ratio of a classified number ofblocks, which is a number of blocks classified into each block category,to a reference number of blocks, wherein the selecting the image outputmode is performed after the classifying each of the plurality of blocksinto one of the plurality of block categories, wherein the plurality ofoutput modes includes at least a first output mode and a second outputmode, the first output mode being prior to the second output mode in theoutput mode order, wherein, in a first determination in the output modeorder, determining whether or not to select the first output mode as theimage output mode based on a ratio of a first classified number ofblocks, which is a number of blocks classified into a first blockcategory, to a first reference number of blocks, which is a total numberof the plurality of blocks of the image, and wherein, in a seconddetermination in the output mode order, determining whether or not toselect the second output mode as the image output mode based on a ratioof a second classified number of blocks, which is a number of blocksclassified into a second block category, to a second reference number ofblocks, which is a value obtained by subtracting the first classifiednumber of blocks from the first reference number of blocks.
 6. Anon-transitory computer readable recording medium storing a computerprogram for causing a computer to perform an image processing method ofselecting, based on data of an image that is acquired, an image outputmode for outputting the image from one of a plurality of output modes,the image processing method comprising: classifying each pixel of theimage into one of a plurality of pixel categories based on the imagedata of each pixel; dividing the image into a plurality of blocks, theplurality of blocks each including a plurality of pixels adjacent toeach other; classifying each of the plurality of blocks into one of aplurality of block categories based on the pixel category of each of theplurality of pixels belonging to each of the plurality of blocks; andselecting the image output mode by sequentially determining, inaccordance with an output mode order allotted to the plurality of outputmodes, whether or not to select one of the plurality of output modes asthe image output mode, based on a ratio of a classified number ofblocks, which is a number of blocks classified into each block category,to a reference number of blocks, wherein the selecting the image outputmode is performed after the classifying each of the plurality of blocksinto one of the plurality of block categories, wherein the plurality ofoutput modes includes at least a first output mode and a second outputmode, the first output mode being prior to the second output mode in theoutput mode order, wherein, in a first determination in the output modeorder, determining whether or not to select the first output mode as theimage output mode based on a ratio of a first classified number ofblocks, which is a number of blocks classified into a first blockcategory, to a first reference number of blocks, which is a total numberof the plurality of blocks of the image, and wherein, in a seconddetermination in the output mode order, determining whether or not toselect the second output mode as the image output mode based on a ratioof a second classified number of blocks, which is a number of blocksclassified into a second block category, to a second reference number ofblocks, which is a value obtained by subtracting the first classifiednumber of blocks from the first reference number of blocks.